In baseball, part of the infielders' job is to cover bases. That is, they stand next to a base in anticipation of receiving the ball thrown from another fielder, so that they may make a play on an opposing baserunner who is approaching that base. On a force play or appeal play, the fielder covering the base stands with one foot on that base.
In general, the first baseman covers first base, the second baseman or shortstop covers second, the third baseman covers third, and the catcher covers home base. On balls to the first baseman, the pitcher or second baseman might cover instead.
With baserunners on first and second, sometimes the shortstop will cover third on a bunt, known as the "rotation play" or "wheelplay."
Depending on the situation, however, various fielders can cover any base, sometimes an outfielder can even cover a base.
In American English, the expression "cover your bases" used outside of a baseball setting means to be thoroughly prepared for something.